DOCUMENT RESUME 



ED 480 226 



CS 512 373 



AUTHOR 

TITLE 

INSTITUTION 
SPONS AGENCY 
PUB DATE 
NOTE 

AVAILABLE FROM 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Johnson, Debra; Foertsch, Mary 

Critical Issue: Monitoring the School Literacy Program. 

North Central Regional Educational Lab., Oak Brook, IL. 

Office of Educational Research and Improvement (ED), Washington, DC. 
2000 - 00-00 
14p . 

North Central Regional Educational Laboratory, Editorial Offices: 
NCREL, 1120 E. Diehl Rd., #200, Naperville, IL 60563. Tel: 800-356- 
2735 (Toll Free). For full text: http://www.ncrel.org/ 
sdrs /areas /is sues /content /cntareas /reading/ li700.htm. 

Guides - Non-Classroom (055) — Opinion Papers (120) 

EDRS Price MFOl/PCOl Plus Postage. 

Educational Objectives; Elementary Education; Instructional 
Effectiveness; *Literacy; Needs Assessment; Program Evaluation; 
*Program Implementation; *Reading Improvement; *Reading Instruction; 
*Writing Improvement; *Writing Instruction 
*Program Monitoring 



ABSTRACT 

This paper discusses monitoring the school literacy program. Teachers 
make many educational decisions about how reading and writing should be taught in the 
classroom. Some teachers makes these decisions intuitively, but a systematic process 
of monitoring is more effective. When teachers monitor the school’s literacy program, 
they track the reading progress of their students, the results of different teaching 
methods, and the value of various materials used to evaluate its effectiveness, and to 
make necessary improvements. Monitoring a school literacy program is a systematic 
process of examining students* reading progress and teachers* instructional strategies 
to improve students* reading and writing achievement. Any monitoring effort has three 
basic components: collecting information on a regular basis, analyzing and evaluating 
that information, and taking action to improve student performance. The careful 
planning of literacy monitoring involves addressing four questions: Why is the program 
being evaluated?; What is being evaluated?; How is the program being evaluated?; and 
How well has it been evaluated? An additional question is implied: At what points 
along the way will the process be monitored? The paper consists of the following 
sections: Issue; Overview; Goals; Action Options; Pitfalls; Different Viewpoints; 
(Illustrative) Cases; and Cojitacts. (NKA) 
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Critical Issue: Monitoring the School Literacy Program 




ISSUE: Teachers make many educational decisions about how reading and writing should be taught in the 
classroom. Some teachers make these decisions intuitively, but a systematic process of monitoring is more effective 
in promoting students' literacy development. When teachers monitor the school's literacy program, they track the 
reading progress of their students, the results of different teaching methods, and the value of various materials used. 
The results of their monitoring activities allow teachers to judge the value of their program— in other words, to 
evaluate its effectiveness— and to make necessary improvements. 

Careful and continuous monitoring is an integral part of an effective literacy program because it enables teachers to 
determine the most effective strategies for teaching reading and writing and because it helps teachers address students' 
learning problems before these problems seriously affect student achievement. The careful planning of literacy 
monitoring involves addressing four questions: Why are we evaluating? What are we evaluating? How are we 
evaluating? How well have we evaluated? An additional question is implied: At what points along the way will the 
process be monitored? 




Overview | Goals | Action Options | Pitfalls | Different Viewpoints | Cases | Contacts | References 



OVERVIEW: Monitoring a school literacy program is a systematic process of examining students' reading 
progress and teachers' instructional strategies in order to improve students' reading and writing achievement. Any 
monitoring effort has three basic components: collecting information on a regular basis, analyzing and evaluating that 
information, and taking action to improve student performance (Richards, 1988). These components may be preceded 
by other activities, such as determining gaps in practice and articulating questions on which to focus the monitoring. 

Monitoring is a continuous process. When teachers monitor the school's literacy program, they keep a watchful eye on 
students' achievement and successes in reading and writing activities. They collect literacy-focused assessment data, 
including standardized tests and alternative assessments (such as samples of students' work and observations of 
students' reading behavior and performance). They also look beyond assessment data to children's attitudes toward 
reading, comments from families, and any other information that sheds light on whether the goals of the literacy 
program are being met. Next, they review and analyze this information to determine the value of various elements of 
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the literacy program and how well these elements foster students' success. Finally, they make adjustments in areas 
that need improvement. Then the cycle of monitoring begins again. 

Effective monitoring practices are necessary for maintaining a quality literacy environment. "All teachers need ways 
to determine what students are learning and the progress they are making. This infomiation provides the basis for 
making decisions, planning instructional activities and experiences, and distinguishing effective from ineffective 
procedures," states Cooper (1997, p. 513). Effective monitoring helps teachers take ownership of the teaching- 
learning process and enables them to implement new strategies to foster students' literacy growth. 

When teachers articulate their literacy goals for students and the types of evidence they need to determine if students 
are successful in meeting those goals, teachers begin to build a foundation for the monitoring process. Monitoring a 
literacy program requires a process for tracking and evaluating current procedures and outcomes. Teachers need to 
know what is working, what isn't, and how well students' efforts are being supported. Program monitoring through the 
assessment, and later evaluation, of teaching strategies and student learning is essential. It aids informed decision 
making in the classroom and school, and it contributes to each teacher's learning. 

Concerns in Monitoring Literacy Programs 

Before the monitoring process can be started, teachers and administrators should be aware of some concerns in 
monitoring literacy programs. More efforts are made to evaluate reading programs than any other curriculum area. In 
spite of these efforts, the evaluation of reading remains shrouded in misunderstanding and misinterpretation. Attempts 
to evaluate reading programs and students' reading achievement often confuse the issue rather than clarify it. There 
are several reasons for this confusion. 

First, evaluation often is set up as an afterthought. A plan is not in place to answer initial evaluation questions before 
the process of teaching reading begins in the classroom. Such evaluation questions include: Why are we evaluating? 
What are we evaluating? How are we evaluating? How well have we evaluated? Instead of being used as a process 
forjudging and improving program quality, evaluation becomes something teachers or administrators are required to 
do in order to satisfy externally determined needs. When this situation occurs, teachers have no ownership of the 
evaluation plan or the generated data. The result is that often no one seems to know why the assessment infomiation 
was gathered or how it can be used to improve instruction. 

Second, the evaluation may be hampered by unclear objectives. If little attention has been devoted to why the 
evaluation is occurring, what is being evaluated may be unclear. Is the evaluation trying to determine how well a 
particular group of students can read? If so, what definition of readin g is being used to determine success? And what 
will be the evidence that the students can read? Perhaps, alternatively, teachers are interested in how well the literacy 
program is succeeding and how well their students as a whole are doing. The evaluation of literacy programs should 
span more than individual children; instead, it addresses questions and judgments of the literacy program itself The 
challenges, however, can remain the same: The definition of reading and the criteria for acceptable reading 
performance may not be clear. To be valid, evaluation demands a standard against which data or infomiation can be 
judged. Without it, no judgment of worth can be made. It is impossible for teachers to determine whether the program 
is working if they have not determined what the program should do. 

A third problem is confusion about the meaning of the term evaluation. Evaluation is the process used to detemiine 
the worth of something; it is an attempt to determine if some product, process, activity, or procedure is of value or is 
satisfactory. Some educators equate evaluation with assessment, although the terms are not synonymous. These temis 
have been used interchangeably for some years, yet their meanings are quite different. "Assessment is the process of 
gathering information about something (getting students to respond), and evaluation is the process of judging that 
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infonnation (judging students' responses) to determine how well individuals are achieving," notes Cooper (1997, p. 
512). Evaluation decisions are based on assessment information and data. Testing, often defined more narrowly as 
standardized testing, is but one way to gather data. When deciding how they will evaluate the literacy program, 
teachers should consider alternative assessment strategies, such as portfolios, projects, teacher observations, surveys, 
and interviews. As teachers clarify their goals for students (based on their definition of reading) and specify what 
evidence they need to determine whether the students are meeting these goals, they realize that there are many ways 
to gather evidence and use it to make a judgment about the literacy program. 

When educators miss the opportunity to answer the first three questions— IfTiy are we evaluating? What are we 
evaluating? How are we evaluating?— XhQy probably will be unable to respond positively to the fourth question. How 
well have we evaluated? The data or information produced may be insufficient or inappropriate to use in evaluating 
the reading program and, if published, could be misinterpreted by the community. (For additional information on 
assessment reporting, refer to the Critical Issue "Re porting Assessment Results .") For example, results of a single test 
that bears no relationship to the literacy program or results of a homegrown survey of attitudes with unknown 
reliability and questionable validity are inappropriate forjudging the overall effectiveness of a literacy program. How 
well the task is carried out really depends on the preparation involved in planning the evaluation. 

Preliminary Activities Before Beginning the Monitoring Process 

Four preliminary activities are necessary before the formal monitoring process can begin. First, the school establishes 
a literacy committee that can direct the monitoring efforts. Second, the faculty is informed by research and best 
practice about literacy, evaluation, and assessment. Third, if gaps between desired practice and ideal practice are not 
known, the school conducts a needs assessment. Finally, the faculty formulates questions to focus the monitoring. 
These activities are followed by the formal monitoring activities: collecting information, organizing and analyzing 
infomiation, and taking action. 

Establishing a Literacy Committee. One of the first activities in monitoring a literacy program is establishing a 
literacy committee whose ultimate goal is to improve the school's literacy program (Patty, Maschof, & Ransom, 
1996). This committee is made up of teachers, administrators, and interested parents and community members, but 
the majority should be school personnel (Patty, Maschof, & Ransom,. 1996). The committee is responsible for 
planning and initiating the monitoring process, gathering data and resources, coordinating with teachers who are 
monitoring the literacy program in their individual classes, leading the decision-making process to chart a new 
direction for the literacy program, and communicating the findings to school personnel and community members. 




Mary Foertsch. fomier director of the Center for Literacy and Assessment at North Central Re gional 
Educational Laboratory, discusses the formation of the school's literacy committee and its important tas ks [101 K 
audio file]. Excerpted from a videotaped interview with Mary Foertsch (North Central Regional Educational 
Laboratory, 2000). A text transcript is available. 



Building a Knowledge Base. The literacy committee is responsible for building a knowledge base about literacy, 
evaluation, and assessment. This knowledge base will be used by the committee and by the rest of the faculty. The 
committee may compile information about reading research, best practices in reading instruction, different ways to 
monitor children's reading progress, and examples of schools that have model literacy programs. It may wish to 
detemiine the definition of reading that drives the school's literacy program and develop an awareness of 
characteristics of outstanding literacy pro grams and characteristics of literacy-rich classrooms . Also important is 
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knowledge of literacy standards for students, such as the lan guage arts standards for grades K-12 and the Continuum 
of Children's Development in Early Reading and Writing for children in preschool through second grade. Finally, the 
knowledge base should include information on the principles of effective literacy assessment as well as effective 
strategies for integrating assessment and instruction in ways that su p port learnin g and ensuring equity with alternati v e 
assessments . 

Conducting a Needs Assessment. After building a knowledge base, the literacy committee may wish to conduct a 
needs assessment for the literacy program. Generally, a needs assessment is a means of examining the overall quality 
of a program, identifying strengths and weaknesses of the program, and recognizing the current needs of students and 
teachers. 

The needs assessment for monitoring a literacy program is a little more focused. Its purpose is to pose questions that 
help teachers identify information they need in order to make instructional or curricular decisions. By answering these 
questions in terms of their own program, teachers first determine what information is valuable and then identify gaps 
between the ideal and reality. The discrepancy between some goal and the reality of a present situation is defined by 
Gottman and Clasen (1972) as a need. The needs assessment identifies these discrepancies and helps provide a focus 
for monitoring and improvement efforts. 

The needs assessment attempts to identify whether teachers and administrators need additional information to analyze 
decisions currently being made about reading instruction. When educators define what information is needed, they 
also clarify the issues to be evaluated. A discrepancy between the information required to make professional decisions 
and the information that is available can be determined by asking a series of questions about educational decisions 
made in the school. 

First are questions about who makes literacy decisions . Answering these questions helps teachers pinpoint 
responsibilities for making specific decisions, such as when children are ready for reading instruction, what reading 
curriculum will be used, and how children's reading progress will be communicated to parents. 

Next are q uestions about information and processes currently used to make literacy decisions. Answering these 
questions enables teachers to determine what specific information about instructional practice and curriculum should 
be known and what information currently is known. For example, teachers' answers at the primary level might reveal 
the strategies and skills that children should acquire before beginning formal reading instruction, teachers' criteria for 
determining these skills, and the background and previous experiences that help prepare children for formal reading 
instruction. Teachers' answers to these questions also help the literacy committee and the faculty evaluate the worth of 
issues they are examining. 

Finally, there are questions about external factors that affect children's reading achievement. Answering these 
questions reminds teachers to consider children's diverse backgrounds and needs. As part of the needs assessment, the 
literacy committee may wish to identify the external factors that affect the success of the literacy program. Factors 
related to students' cultural backgrounds , second-language learnin g, socioeconomic status , and s pecial needs , as well 
as factors related to teacher effectiveness have been found to affect students' reading achievement (Allington & 
Cunningham, 1996; Au, 1993; Bialystock, 1991; Foertsch, 1998; Garcia, E., 1994; Heath & Mangiola, 1991; Snow, 
Bums, & Griffin, 1998). Identification of these factors helps ensure that all aspects of a literacy program are included 
in the monitoring process and that steps eventually will be taken to address weak areas. 

Teachers' answers to all these questions help the literacy committee clarify the information currently used to make 
those decisions and the gaps in information that may exist. The level of information available becomes critical. 
Through this questioning procedure, the literacy committee can identify what available information is appropriate or 
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necessary for making professional decisions about the literacy program and determine if that information currently is 
being used. If current information is different from what ideally is needed, there is a discrepancy. Any discrepancy 
indicates a need to collect better information and improve program monitoring. 

The needs assessment may be conducted through questionnaires, surveys, or worksheets for school staff. The literacy 
committee is responsible for collecting the responses or data, assisting the faculty in analyzing and prioritizing the 
concerns, and helping teachers formulate questions based on those concerns to focus the monitoring. 

Formulating Questions to Focus the Monitoring. After analyzing the needs assessment, the literacy committee 
works with teachers to formulate specific questions for the monitoring process. These questions help teachers 
determine an area of focus before they begin collecting data on children's literacy performance. The area of focus 
should involve teaching and learning and be within the control of the school (Mills, 1999; Calhoun, 1994). Farr and 
Prichard (1989) state, "Formulation of clear and explicit evaluation questions is the most important step in planning 
and implementing effective assessment of the reading program" (p. 164). When formulating questions, the literacy 
committee can follow rules of thumb for determining focus questions . 

The focus questions are based on the gaps found in the needs assessment and should emphasize curricular and 
instructional issues. For example, the needs assessment may indicate that not enough information is available to 
determine if the primary literacy program is developmentally appropriate, or if the upper-elementary students have 
the skills and strategies required to make text-based inferences. Examining these needs will enable teachers to 
formulate specific evaluation questions on which to focus. In this situation, an evaluation question for the primary 
program might be: Do the activities we use with emergent readers in first grade help them meet the literacy goals 
established for grade-level achievement? For the upper-elementary program, an evaluation question might be: Does 
our instruction help the students make increasingly sophisticated connections between sentences in a passage and use 
evidence from the text to draw logical conclusions? 

For each question that is formulated, teachers need to determine two types of information: what is and what should 
be. What is relates to what the students currently are able to do in the classroom. What should be relates to what the 
students should be able to do, according to a standard or expectation of student performance. To be successful, the 
monitoring process should have a standard against which data or information can be measured. Without a standard, it 
is impossible for teachers to determine how well students are achieving. 

Formal Monitoring Activities 

After the focus questions have been determined, it is the teachers' turn to drive the monitoring process. Teachers now 
assess the literacy curriculum, instructional practices, and students' literacy performance. First, they determine what 
information is valuable in answering the focus questions. Then, they collect data and pertinent information in their 
classrooms. Next, they organize and analyze that data (with the assistance of the literacy committee). Finally, they 
take action to improve students' literacy achievement. 

Collecting Information. Data collection must relate specifically to the questions that were formulated to focus the 
monitoring (Farr & Pritchard, 1989; Padak & Padak, 1994). The challenge is ensuring that data is collected on 
"enough important variables to be certain all major and relevant considerations are included in the evaluation," note 
Worthen and Sanders (1987, p. 233). The information should provide a clear picture of how the school environment 
affects student learning; it also should help identify factors affecting the program that are within the control of the 
school (Calhoun, 1994). 

Teachers must identify the information needed to respond to the questions. They determine what concrete and 



http://www.ncrel.org/sdrs/areas/issues/content/cntareas/reading/li700.htm 



anecdotal information will be collected about students and what information will be collected about the environment 
(Calhoun, 1994). For example, if the needs analysis indicates that a focus of the monitoring process should be 
children's readiness to read and how to begin formal instruction, teachers can collect information on readiness skills 
specified in the curriculum and standards being used, experiences that children have had with literacy prior to 
entering the class, and approaches currently being used to determine children's literacy skills and awareness levels. 
"Not all information needs can be identified in advance, but surely many can and should be. Identifying information 
needs at the outset ... offer[s] the advantage of following a plan for collecting information that continues to be viewed 
as important throughout the evaluation," state Worthen and Sanders (1987, p. 232). 

Three sources of information help provide an accurate picture of the school literacy program: existin g archival 
sources of data (such as student grades and standardized test results), conventional sources of data (such as samples of 
students' work, observations, and anecdotal records), and inventive sources of data (such as portfolios, exhibits, and 
videotapes). Teachers may wish to collect archival data first and then move to conventional and inventive data 
sources. The use of multiple sources of data provides a more complete picture of students and the learning 
environment than does a single source (Calhoun, 1994; Padak & Padak, 1994). Because a single data source is 
limited, it may provide inaccurate information and lead to false conclusions (Calhoun, 1994; Worthen & Sanders, 
1987). These sources can be plotted along a continuum, with existing sources at one end and inventive sources at the 
other. As one moves across the continuum, the collected information gains depth; as a result, additional time is 
necessary to analyze and interpret the data (Calhoun, 1994). 

Teachers can keep in mind that data collection should occur regularly, should be as simple as possible, and should 
promote collective ownership of data. Following these suggestions will improve the quality of data collected. Also, 
the data-collection process may need to be monitored until procedures for collecting become established among the 
teachers who are collecting the data (Calhoun, 1994). 

Organizing and Analyzing Information. After collecting the data, teachers work with the literacy committe to 
organize it in a clear format. The manner of organization depends upon whether the data is quantitative or qualitative . 
Organization of quantitative data involves the use of charts or tables to provide clear information for short-term and 
long-term decision making. Organization of qualitative data involves working with words rather than numbers, so this 
data can be arranged in paragraph descriptions or summaries. If there is a great deal of qualitative information, it 
initially can be sorted into piles according to questions. Data not directly related to questions should be set aside 
(Padak & Padak, 1994). Some qualitative information may need to be reread or revisited several times (Mills, 1999). 

If various staff members have collected information relating to a specific question, the data should be shared when it 
is being organized. For example, classroom teachers may collect information on students' literacy experiences outside 
of school, office staff may collect information on attendance or grades, and the reading specialist may review the 
reading curriculum to determine skills and concepts to be taught. 

After organizing the data, teachers and the literacy committee analyze it. Using guidelines for data analysis , they 
study the data and try to determine patterns, trends, and areas of strength and weakness. They also compare data from 
various sources and try to formulate conclusions, such as possible causes or reasons for students' performance. 
Calhoun (1994) states, "Time needs to be provided for the faculty to study, discuss, and question the data as a 
professional collective; to determine priority area(s) for action; and to decide what can be celebrated" (p. 80). 

Data can be organized and analyzed on three levels: school, grade level, and classroom, but "it is the grade- and class- 
level data (and for the teacher it is individual student data) that provide the primary diagnostic data needed to guide 
schoolwide collective action," notes Calhoun (1994, p. 81). An example of data analysis provides a glimpse of one 
school's efforts at analyzing and interpreting data. 
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Taking Action. The final step of monitoring a literacy program is taking action to improve student performance. 
During this step, teachers apply what they have learned and develop an action plan. The information provided through 
data analysis and the information collected for the knowledge base help teachers focus collective action on those 
options most likely to yield positive benefits for students. The action plan should include both short-term and long- 
term actions. 

The selected actions are influenced by the school's literacy goals for all students and should be based on best teaching 
practices, research, and cument literature on literacy . Actions should focus on curriculum and instruction and should 
be specific. Because teachers have been involved in the ongoing monitoring process, they understand the importance 
of taking action and are willing to implement specific changes into their daily lessons and teaching strategies. The 
action plan needs to include techniques for studying implementation of new curricula and methods of instruction 
(Calhoun, 1994). Tracking the effectiveness of the selected actions continues the monitoring cycle. 

Ongoing monitoring of the school literacy program is an effective means to ensure the continuing literacy 
development of students. Use of the Literacy Program Evaluation Tool , which outlines the steps in the monitoring 
process, provides guidelines for building-level personnel to examine students' reading progress and teachers' 
instructional strategies. When teachers are involved in the monitoring process, they stay informed about students' 
literacy achievements in relation to school goals. They also are more willing to implement improvement strategies in 
the classroom. The end result is a more effective literacy program for all students. 



GOALS: 

• Monitoring of the school's literacy program leads to the implementation of best teaching practices. 

• Monitoring of the school's literacy program provides opportunities for staff to reflect on current practices. 

• Monitoring of the school's literacy program is ongoing and integrated with instruction to support learning. 

• Monitoring of the school's literacy program occurs as an inquiry process. 

• Information about students' reading progress, strengths and weaknesses of the literacy program, and 
instructional effectiveness is gathered from a wide variety of sources, not just from standardized tests. 





ACTION OPTIONS: The literacy committee and classroom teachers can take the following steps to 
monitor and evaluate the school's literacy program. 



Literacy Committee: 

• Work with teachers to build a knowledge base about literacy by reviewing current research, best practice, and 
examples of schools with outstanding literacy programs. 

• Work with teachers to identify literacy standards for students, such as the language arts standards for grades K- 



http ://www.ncrel . org/sdrs/areas/i ssues/content/cntareas/reading/li 700 . htm 



12 and the Continuum of Children's Development in Early Readin g and Writing for children in preschool 
through second grade. 

• Work with teachers to identify factors in the school or district that affect student achievement in learning to 
read. 

• Conduct a needs assessment to identify areas of strength and concern in the literacy program. 

• Work with teachers to formulate focus questions to guide the monitoring process. 

• Hold teacher study group s, analyze school records, conduct surveys and interviews, or hold a community forum 
on students' literacy achievement. Collect information to answer identified questions. 

• Set up timelines listing steps to be taken during data collection, and assign roles to make sure the appropriate 
information is collected in a timely manner. 

• Work with teachers to develop monitoring techniques for their classrooms. 

• Provide encouragement and information to help teachers in their monitoring activities. 

• Help teachers analyze and summarize data. Organize quantitative data into simple frequency tables or charts 
with information arranged by class, grade level, or school; create tables by simply counting instances, events, 
and artifacts. Organize qualitative data in a clear, easily read format. Present the findings to school staff 

• Work with teachers to develop an action plan for implementing new literacy strategies in the classroom. 

Teachers: 

• Ensure that the literacy program for the younger children focuses on addressing the literacy needs of emergent 
and early readers . 

• Acknowledge that the most valuable form of reading assessment reflects current understanding about the 
reading process . 

• When considering the types of data to collect, be sure to select or desi g n assessments that elicit establish ed 
outcomes . 

• Use many forms of student assessment, such as portfolios , anecdotal records , and performance assessmen t. 
Develop methods for observing and recordin g and knowledge of other literacy assessment techniques to ensure 
greater accuracy of achievement levels. 

• With the help of the literacy committee, organize and analyze the data collected. Look for patterns, trends, and 
areas of strength and weakness. Determine explanations for these results. 

• Identify main areas of concern and develop an action plan to address them. 

• Determine appropriate actions to improve the school's literacy program. Consider adopting new instructional 
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strategies to improve students' literacy performance. 

• Review current research and literature on literacy— such as Im proving the Readin g Achievement of America's 
Children: 10 Research-Based Principles . State of the Art: Transforming Ideas for Teaching and Learning to 
Read. Exemplary Reading Programs in Illinois Public Schools . Wlrat Does Research Say About Readin g?. 
Educators: What You Can Do . and Start Early. Finish Strong: How to Help Every Child Become a Reader — to 
ensure that the actions are based on best teaching practices. 

• Devise assessments to monitor the success of the actions. 

• Implement the actions in the classroom. Continue to monitor the results of these actions. 



IMPLEMENTATION PITFALLS: Even though monitoring the school's literacy program is of critical 
importance, some teachers who are faced with the day-to-day workload of preparing lessons, instructing, keeping 
records, correcting papers, and disciplining may consider ongoing monitoring to be a luxury for which they do not 
have time. The principal or superintendent should emphasize the importance of monitoring the literacy program and 
ensure that teachers have adequate time to collect and interpret data. (For additional information on providing time for 
teachers, refer to the Critical Issue " Finding Time for Professional Development ."! 

Some educators may suggest using data already available instead of collecting new data. In such situations, the 
questions for monitoring can be manipulated to focus only on data that is readily available. Although this approach 
may appear to be time-efficient, it will not yield reliable or valid results (Worthen & Sanders, 1 987) and will not be as 
helpful in meeting the goal of improved student learning. Teachers need to keep in mind the goals of the monitoring 
process as they collect new data. 

Sometimes decisions are based on limited data or no data. Instead of making a premature analysis, educators need to 
be patient and allow the evaluation process to fully unfold (Mills, 1999). It also is important for educators to avoid 
over-reliance on perceptual data, such as surveys and attitudinal measures. Although the use of surveys and 
questionnaires provides some useful information, the most important information is based on actual student 
performance in the classroom. Calhoun (1994) advises educators to "collect behavioral data whenever possible ... and 
keep your initial surveys short and of high quality" (p. 56). 

The school may have policies that restrict data collection or limit data-collection procedures. Such restrictions may 
occur in the form of teacher contractual agreements, state requirements and approvals, rights of privacy, or district 
policy. If restrictions are identified early in the planning process, accommodations may be worked out upfront and 
save valuable time later (Worthen & Sanders, 1987). 

Some teachers and principals may not know how to go about collecting data, or they may have set ways of analyzing 
and interpreting the data (Patty, Maschof, & Ransom, 1996). This situation can be addressed by providing staff with 
professional development on data-collection strategies so they understand the process of monitoring and the value of 
data collected. Professional development activities may take the form of reviewing monitoring plans from other 
schools and discussing current literature on monitoring programs to determine which methods will work best for the 
school's literacy program. 

Teachers and administrators may express reluctance to begin program monitoring simply because they do not know 
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where to begin. Although they sense why they should be monitoring student progress and understanding assessment 
strategies, they may be unable to translate philosophy into action. They also may be unsure of how to determine a 
focus that would make the monitoring process meaningful and manageable. Under those circumstances, a needs 
assessment can play an important role in getting started. 

Mills (1999) notes that obstacles such as "lack of resources, resistance to change, reluctance to interfere with others' 
professional practices, reluctance to admit difficult truths, and the challenges of finding a forum to share what you 
have learned" may impede teachers from taking action to effect change. 

DIFFERENT POINTS OF VIEW: Some administrators and educators lightly regard program 
monitoring because they mistrust data. Some have been stung by data gathered by someone else. Many are confused 
by the jargon of the evaluator. Others are discouraged because past results often seem inconclusive. Still others insist 
that because teaching reading is more an art than a science, it cannot be scientifically measured. 

Some parents and educators believe that standardized tests are the most valid representation of the value of a literacy 
program and the bottom line in measuring students' reading performance. They may not consider alternative 
assessment data to be valuable to collect and evaluate during the monitoring process. 




ILLUSTRATIVE CASES: 



• A Study of Reading Practices. Instruction, and Achievement in District 31 Schools is an NCREL research study 
that focuses on the literacy program at West Northfield District 31 in Illinois. 

• Lafayette School, in Chicago, monitored its literacy program with the help of North Central Regional 
Educational Laboratory, [under development] 



CONTACTS: 




Center for the Improvement of Early Reading Achievement (CIERA) 

University of Michigan, School of Education 

610 E. University Ave., Room 1600 SEB 

Ann Arbor, MI 48109-1259 

(734) 647-6940; fax (734) 763-1229 

Contact: Elfrieda Hiebert, Director 

E-mail: ciera@umich.edu 

WWW: htt p://www.ciera.or g/ 

ERIC Clearinghouse on Reading, English, and Communication 
Indiana University 



O 

ERIC 
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Smith Research Center, Suite 150 
Bloomington, IN 47408-2698 

(800) 759-4723 or (812) 855-5847; fax (812) 855-4220 
WWW: http://www.indiana.edu/~eric_rec/ 

International Reading Association 

800 Barksdale Road 

P.O. Box 8139 

Newark, DE 19714-8139 

(302) 731-1600; fax: (302) 731-1057 

WWW: htt p://www.reading.or g/ 

National Center for Research on Evaluation, Standards, and Student Testing (CRESST) 

University of California at Los Angeles 

Graduate School of Education and Information Studies 

300 Charles E. Young Drive North 

Los Angeles, CA 90095-1522 

(310) 206-1532; fax 310-825-3883 

WWW: http://cresst96.cse.ucla.edu/ 

National Council of Teachers of English 
1111 West Kenyon Road 
Urbana, IL 61801 

(211) 328-3870 or (800) 369-6283; fax (217) 328-9645 
E-mail: webmaster@ncte.org 
WWW: htt p://www.ncte.or g/ 

National Institute for Literacy 
1775 I St. N.W., Suite 730 
Washington, DC 20006-2401 
(202) 233-2025; fax (202) 233-2050 
Contact: Andrew Hartman, Director 
E-mail: jbehroozi@nifl.gov 
WWW: htt p://www.nifl.gov/nifl/ 

National Research Center on English Learning and Achievement 
Albany Institute for Research in Education 
University at Albany, ED-B9 
1400 Washington Ave. 

Albany, NY 12222 
(518) 442-5026; fax (518) 442-5944 
Contact: Janet Angelis, Associate Director 
WWW : htt p://cela.albany.edu/ 
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This Critical Issue was written by by Debra Johnson, a freelance writer who also is a multiage resource teacher at 
Lincoln School in Mundelein, Illinois, and Mary Foertsch, former director of the Center for Literacy and Assessment 
at North Central Regional Educational Laboratory. 

Date posted: 2000 
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